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SUMMARY 


A joint experiment to investigate simulator validation and cue fidelity was con- 
ducted by the Dryden Flight Research Facility of NASA Ames Research Center (Ames- 
Dryden) and NASA Langley Research Center. The primary objective was to validate the 
use of a closed-loop pi lot-vehicle mathematical model as an analytical tool for 
optimizing the tradeoff between simulator fidelity requirements and simulator cost. 
The validation process includes comparing model predictions with simulation and 
flight test results to evaluate various hypotheses for differences in motion and 
visual cues and information transfer. A group of five pilots flew air-to-air 
tracking maneuvers in the Langley differential maneuvering simulator and visual 
motion simulator and in an F-14 aircraft at Ames-Dryden. The simulators used mo- 
tion and visual cueing devices including a g-seat, a helmet loader, wide field-of- 
view horizon, and a motion base platform. The acquisition and preparation of the 
flight test data for analysis are described. Subjective results of pilot question- 
naires obtained from the flight experiment also are presented. 


INTRODUCTION 


For many years, NASA has been involved in simulator technology, specifically in 
addressing the issues of simulator design. Two factors that must be considered in 
simulator design are fidelity and cost. For example, performing desktop calcula- 
tions results in little fidelity but the cost is low. On the other hand, flight 
testing provides maximum fidelity but is expensive when highly sophisticated air- 
craft are used, various levels of fidelity between that afforded by desktop calcu- 
lations and by flight test are provided by simulation. In this instance, the level 
of fidelity is dependent on the complexity of the simulation, which in turn is re- 
lated to cost. The cost factor raises two important questions: (1) How much fidel- 

ity is sufficient, and (2) Can the degree of fidelity be quantified with precision? 

NASA Langley Research Center is attempting to answer these questions by sponsor- 
ing the development of an analytical closed-loop pilot-vehicle system model of the 
Langley real-time simulation system (Ashworth and others, 1983; Baron, 1976, 1980; 
Baron and Kleinman, 1971; Baron and others, 1970; Baron and Muralidharan, 1980; 

Baron and others, 1978; Baron and others, 1980; Parrish and Ashworth, 1976). This 
analytical system model combines a multiaxis model of the human pilot with detailed 
models of the components of a simulator. The system model is used to generate ana- 
lytical predictions for comparison with the data from an in-simulator experiment. A 
joint experiment between the Dryden Flight Research Facility of NASA Ames Research 
Center (Ames-Dryden) and NASA Langley Research Center extends this effort by includ- 
ing flight data as well as simulator data in the comparison. The Ames-Dryden role 
is to provide the flight data for the experiment and to develop a method for eval- 
uating these data for closed pilot-in-the-loop analysis. The system model can then 
be used as a tool to analyze both hardware and software design by changing the hard- 
ware and software models. The effects of these changes on the task performance pro- 
vide the quantitative data for the evaluation of tradeoffs between fidelity and cost 

A description of the experiment with emphasis on the acquisition of flight 
data and the method developed for analyzing flight test data for pilot-in-the-loop 



analysis is the scope of this report. A summary of the preliminary subjective results 
of pilot questionnaires obtained from the flight experiment is also presented. 
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NOMENCLATURE 

x-y coordinate points on reticle, mils 

aircraft type, field name 

average 

calibration marker on frame of 2-sec film segment 
differential maneuvering simulator 
right rudder position, deg 

end or stop marker on frame of 100-sec film segment 

film magazine number, field name 

flight number, field name 

frame sequence number, field name 

frequency, rad/sec 

film reference point 

film scoring sample rate, samples/sec 
true altitude, ft 
identification code, field name 
initial start time, HR, MIN, SEC, MSEC 
x coordinate of TLWD, field name, counts 

y coordinate of TLWD, field name, counts 

x coordinate of PIPD, field name, counts 

y coordinate of PIPD, field name, counts 

x coordinate of TRWD, field name, counts 

y coordinate of TRWD, field name, counts 

x coordinate of TARGD, field name, counts 
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KTY 


OSDM 

MDR 

MDX 

MDY 

PCM 

PMDR 

PMDX 

PMDY 

PIPD 

PREFX 

PREFY 

PRP 

rad 

RANGE 

RD 

RDM 

RUNUM 

S 

SAS 

SP 

SPX 

SPY 

SQRT 

TARGD 

TBX 


y coordinate of TARGD, field name, counts 
optical sight drive motor, input signal, volts 
target radial tracking error, mils 
target azimuth tracking error, x coordinate, mils 
target elevation tracking error, y coordinate, mils 

pulse-code modulation • ' 

pipper radial position from pipper reference, mils 
pipper azimuth position, x coordinate, mils 
pipper elevation position, y coordinate, mils 

pipper image position (displacement) from film reference point, counts 

pipper azimuth reference, x coordinate, counts 

pipper elevation reference, y coordinate, counts 

pipper reference point 

radian 

target range, ft 

reticle diameter, uncalibrated, counts 
reticle diameter, calibrated, mils 
run number, field name 
start marker on frame 
stability augmentation system 

apparent wing span of target aircraft, counts on 100-sec film segment 
apparent wing span of target aircraft, x coordinate, counts 
apparent wing span of target aircraft, y coordinate, counts 
square root 

target image position (displacement) from film reference point, counts 
target azimuth bias, x coordinate, counts 
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TBY 

TLWD 

TRWD 

VMS 

WGSPN 

x,y 


target elevation bias, y coordinate, counts 

target left wingtip image position (displacement) from film reference 
point, counts 

target right wingtip image position (displacement) from film reference 
point, counts 

visual motion simulator 

actual wingspan of target aircraft, ft 

conventional azimuth and elevation axis coordinates, respectively 


DESCRIPTION OF EXPERIMENT AND EQUIPMENT 


To validate the analytical closed-loop pilot-vehicle system model, mathematical 
model predictions are compared with simulation and flight test results. Various 
hypotheses are evaluated to determine differences in pilot performance resulting 
from motion cues, display cues, and information transfer between simulation and 
flight. The task designed for this experiment consists of flying air-to-air track- 
ing maneuvers using an F-14 aircraft as the pursuit plane and a T-38 or an F-104 air- 
craft as the target plane. All maneuvers were performed at an altitude of 10,000 ft 
while tracking in a 3-g turn or while tracking with wings level. Target range was 
maintained at approximately 800 ft. The target aircraft was flown in a precise 
steady-state 3-g turn or in wings-level flight. A controlled disturbance was pro- 
vided in the optical gunsight display (fig. 1(a)) by moving the reticle (fig. 1(b)) 
parallel to the vertical (y-xis) direction in response to the sum of 11 generated 
sine waves (harmonics), with fixed amplitude and random initial phasing between sine 
waves, wings-level maneuver provided data to check out the analytical pilot-vehicle 
mathematical model. The steady-state 3-g maneuver provided the primary data for 
accomplishing the experimental objectives. 

A group of 5 evaluation pilots, consisting of 2 NASA pilots, 2 U.S. Navy pilots, 
and 1 Grumman Aerospace Corp. pilot, participated in the experiment. Each pilot flew 
tracking maneuvers in the Langley differential maneuvering simulator (DMS) and visual 
motion simulator (VMS) (figs. 2 and 3, respectively) and in the F-14 aircraft at 
Ames-Dryden. The simulators used two mathematical models and motion and visual cue- 
ing devices including a g-seat (fig. 4), a helmet loader (fig. 5), a head-up display, 
and platform motion. These devices are described in Ashworth and Kalhbaum (1973); 
Ashworth and McKissick (1978); Ashworth and others (1983, 1984); Parrish and others 
(1973a, 1973b). The F-14 configuration was selected because the aircraft had been 
used in extensive parameter identification flights during a previous experiment and 
because the simulator was believed to be highly representative of the actual aircraft 
throughout the flight envelope. Also, the F-14 aircraft was available as it was 
based at the Ames-Dryden facilty. 

The pursuit or tracking aircraft used for this experiment was the F-14 aircraft, 
production model IX (fig. 6). The aircraft was flown with the flight-test nose boom 
installed and with pitch, roll, and yaw stability augmentation system (SAS) on. A 
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10-bit data recording system with pulse-code modulation (PCM) was installed in the 
F-14 aircraft and used to record air data, control input, control surface deflection, 
and aircraft response parameters. These data were also recorded on magnetic tape • 
aboard the aircraft. Selected parameters were recorded and monitored in real time at 
the ground station. 

The photographic instrumentation used to obtain tracking data consists of a pro- 
grammable optical gunsight and a high-quality 16-mm movie camera. The optical gun- 
sight and camera is mounted as a single unit above the F-14 cockpit instrumentation 
panel, as shown in figure 1 (a) . The camera is activated by the trigger first-detent 
position on the pilot control stick. A second-detent position on the control stick 
can be used for simultaneous activation of an event marker on the data system and a 
light event marker (streak) on the top edge of the camera film. This data event is 
used in postflight analysis to time correlate film data with PCM data. The movement 
of the reticle image along the y axis is provided by rotating the combining glass on 
the optical gunsight by a servomotor drive. The signal that drives the servomotor is 
produced by a drive electronics package mounted in the rear cockpit and programmed to 
generate the same drive signal as in the Langley simulators to the sum of 11 sine 
waves (harmonics) with frequencies and amplitudes as shown in table 1 . 

The optical gunsight is operated by a power switch, a trim control knob, and a 
reset switch located above the master generator panel on the pilot's right-hand con-'c 
sole, as shown in figure 7(a). The power switch is used to turn on the power to :: 
both the optical gunsight and the movie camera. The reset switch is used to acti- 
vate the drive electronics. When the reset switch is turned on, the combining glass 
(fig. 1(a)) moves to a reset position with the reticle at a fixed pipper depression 
angle, as shown in figure 7(b). This depression angle, set at 5° for the experi- 
ment, can be varied with the trim control knob. When the reset switch is turned off, 
the drive electronics are activated to start the reticle oscillating about the pipper 
depression angle to the sum of the 11 harmonics. Initially, the amplitude of oscil- 
lation is very small, ramping in to a maximum amplitude of ±4.5° for 20 sec. 

The target plane for the experiment was either a T-38 or an F-104 aircraft. 

These aircraft did not have flight test instrumentation aboard and were used only 
to provide a target for the tracking aircraft to provide tracking errors. 


TEST PROCEDURE 


The experiment was divided into three phases. During the first phase, the pilots 
flew the tracking maneuvers in the DMS and VMS. For the second phase, the pilots 
repeated the tracking maneuvers with the F-14 pursuit aircraft at Ames-Dryden. 

During the third phase, the pilots returned to Langley to fly the tracking maneu- 
vers again in the DMS and VMS. Each pilot was given a questionnaire to be completed 
after flying the DMS, VMS, and after actual flight. The pilots used the Ames-Dryden 
developed simulator pilot rating scale (table 2) to evaluate the simulators after all 
flying was completed (Szalai, 1981). The questionnaire that was used for the in- 
flight phase is presented in appendix A; 
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Simulator Test 


During simulation, the pilots acquired tracking data in the DMS and VMS with a 
combination of visual and motion cueing. The configuration test conditions used 
for these sessions are summarized in table 3. Prior to each simulator session, each 
pilot was briefed on the type of tracking maneuvers to be . flown and cueing devices 
used. The pilot flew either seven 3-g tracking maneuvers or three wings-level 
tracking maneuvers at the specified test configuration. The wings-level maneuvers 
were flown only in the VMS at selected test configurations. 

For the 3-g maneuvers using the nonlinear model that had six degrees of freedom, 
the simulator was initialized at straight-and-level flight from which the pilot 
started tracking the target with a windup turn to 3 g. After the simulation was 
stabilized at 3 g for 30 sec, the reticle drive electronics were activated and 
allowed 20 sec to ramp in. Data were simultaneously recorded while the pilot con- 
tinued to track the target for 90 sec. For the 3-g maneuvers with the linear model 
that had three degrees of freedom, the simulator was initialized to have the pilot 
start tracking in a 3-g turn. After the simulation was stabilized at 3 g for 
20 sec, the reticle drive electronics were activated and allowed 20 sec to ramp in. 
The pilot continued to track the target for 90 sec as the data were being recorded. 
Wings-level maneuvers were flown by having the pilot stabilize the simulation in 
straight and level flight for 10 sec before activating the reticle drive elec- 
tronics. After the drive electronics were allowed 20 sec to ramp in, data were 
simultaneously recorded while the pilot continued to track the target for 90 sec. 


Flight Test 


During the flight test phase of the experiment, each pilot flew two flights in 
the F-1 4 aircraft to acquire a total of 10 data tracking maneuvers (3 maneuvers with 
wings level and 7 maneuvers at 3 g). During each flight, five film magazines to 
film the maneuvers were carried aboard the aircraft. Thus the number of data maneu- 
vers per flight was limited to five (one tracking maneuver per magazine). The pilot 
flew three wings-level and two 3-g tracking maneuvers during the first flight. For 
the second flight, the pilot flew five 3-g tracking maneuvers. Prior to flying the 
first tracking maneuver for data film recording, the pilot flew a minimum of two 
practice maneuvers. 

The pilot flew the tracking maneuver by first establishing the target range at 
the flight condition and checking that the reticle drive electronics were in reset 
mode. The pilot then notified the ground station which film magazine number would 
be used to film the maneuver and verified that the correct depression angle was set 
on the optical sight. A short film burst (2 sec) was made to obtain a reticle 
calibration. The pilot notified the ground station when he activated the reticle 
drive electronics and began tracking the target. To allow the drive electronics to 
fully ramp in, filming of the maneuver did not start until the ground station noti- 
fied the pilot that 20 sec had elapsed. The pilot depressed the trigger first- 
detent position on the control stick to operate the camera while continuing to 
track the target for 100 sec. While tracking, the pilot depressed the trigger 
second-detent position for 2 sec to activate the data event and film light event 
at two designated times specified from the ground station. The pilot terminated 
the maneuver after notification from the ground station that 100 sec had elapsed. 
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PREPARATION OF FLIGHT DATA FOR ANALYSIS 


Figure 8 is a flow diagram of the preparation of the flight data for analysis. 
The raw flight data were obtained during flight from two sources, the movie film and 
a PCM flight tape. The PCM data were reduced to provide control inputs, control 
surface deflections, corrected aircraft response parameters, air data, and engine 
parameters; these data were then stored in a PCM data file at a rate of 50 samples/ 
sec. The film data were reduced to provide azimuth and elevation tracking errors, 
target range, and pipper position along the y axis; these data were stored in a 
tracking data file at a rate of 12 samples/sec. Both PCM and tracking data files 
were then time-correlated and merged into a single file at a rate of 12 samples/sec 
(Maine, 1981). This combined data file was used to generate digital listings and 
time history sets for quick analysis. The data were also stored on a disk pack at 
Ames-Dryden for in-house follow-on analysis and on magnetic tape. The magnetic tape 
was sent to Langley. The procedure developed for scoring the film and for computing 
the tracking and PCM data to create a combined data file was essential for corre- 
lating simulator and flight test pilot tracking performance. This is the first time 
that tracking data from gunsight camera film was scored and combined with PCM param- 
eters in this manner at Ames-Dryden. 


Film Scoring 

All movie film was scored at the U.S. Air Force film reading facility at 
Edwards Air Force Base. Ames-Dryden received the raw data on computer cards. 

The data were further processed with an in-house computer program to compute 
the tracking parameters. 

For film scoring, the film image is projected on a large screen which has movable 
horizontal and verticle crosshairs. The lens of the projection system allows the 
entire film image, including the film sprocket holes, to be projected on the screen. 
Figure 9 shows a typical film frame with the reticle, pipper, target aircraft, film 
reference point (FRP), scoring axes (x, y) , and basic scoring parameters. The scor- 
ing parameters, measured in counts, are the image positions or displacements from 
the film reference point for the pipper (PIPD), the target (TARGD), the target left 
wingtip ( TLWD ) , and the target right wingtip (TRWD). 

The movie camera is mounted on its side in the aircraft so that the camera y 
axis or elevation is oriented along the frame width and the x axis or azimuth is 
oriented along the frame height. The frame, as shown in figure 9, has been rotated 
90° to position the positive x axis to the right and the positive y axis upward. 

The film motion is such that the frame advances along the positive x axis. During 
the tracking run, the reticle and the target aircraft image move on the frame as the 
film is advanced. The reticle moves up and down along the y axis over approximately 
50 percent of the frame, while the target image moves about the pipper of the reti- 
cle as the reticle moves up and down the frame. The crosshairs can be positioned at 
any point on the screen, and the x and y coordinates can be read out in counts from 
a specified film reference point. 

The film for each tracking run consists of two time segments: a 2-sec segment 

of filming the reticle in a fixed position and a 100-sec segment of filming the 
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target aircraft during the tracking run. The 2-sec segment is used for establish- 
ing a pipper reference point and reticle calibration. The 100-sec segment is used 
for obtaining the tracking errors, pipper position, and target range. The film is 
marked to aid the technician in scoring the film. The letter C is marked on a frame 
in the 2-sec segment to denote where the film is to be read for calibration. The 
letters S and E are marked in the 100-sec segment to denote where the film scoring 
is to be started and ended. 

After the film is loaded on the projector and focused to provide the best image, 
the scoring technician calibrates the film by advancing the film to the frame with 
the C marker in the 2-sec segment. The crosshairs are placed over the film. refer- 
ence point (fig. 9) and the coordinates are zeroed. (All film readings are scored in 
terms of x and y coordinates in units of counts in relation to this film reference 
point.) On the frame following that marked with C for calibration, the crosshairs 
are placed on 60 mils at point A, 60 mils at point C along the x axis, on the pipper 
at point B, and on 60 mils at point D along the y axis. The x and y coordinates are 
recorded at each point, as indicated in figure 9. The process is repeated twice 
with the film advanced one frame each time. This completes the film calibration. 

The film is than advanced to the S-marked frame in the 100-sec segment. The 
first frame to be scored is the one immediately following the S-marked frame. The 
pipper and target are scored for this frame and every alternate frame thereafter 
until the end or E-marked frame. In addition, the left and right wingtips of the 
target aircraft are scored for the first frame after the S-marked frame and every 
twenty-fourth frame thereafter to the E-marked frame. For scoring of the target, 
the crosshairs are positioned at a point midway between the tail engines (fig. 9). 

If the target has a single engine, the crosshairs are placed over the center of 
the engine exhaust area. If the target is in a nose-high attitude in which the 
nose could be seen (such as in a high-g turn), the target is scored by placing 
the crosshairs over a midfuselage point along a line between the nosetip and the 
tail engine. 

Before being received by Ames-Dryden, all data were recorded on magnetic tape 
and transferred to computer cards in the format shown in table 4. In addition to 
the basic scoring parameters, secondary parameters were recorded to identify the 
specific tracking run. These secondary parameters include identification of air- 
craft type, flight number, film magazine number, run number, and frame sequence 
number. The frame sequence number is assigned a zero for the first scored frame and 
is incremented sequentially throughout the run. The identification code is used to 
indicate whether the data are for calibration, whether the quality of the data was 
bad or good, and whether the film light event marker was on or off. The film light 
event marker is indicated by a thin light streak at the top edge of the frame and is 
used to show when the pilot has activated the data event marker using the trigger 
second-detent position on the control stick. The only secondary parameters that 
change during the run are the identification code and the frame sequence number; 
the others remain constant. 

A portion of the scored film data for a typical tracking run is presented in 
table 5. The namelist parameters at the top of the tabulation are inputs for 
computing the primary tracking parameters and are discussed in the next section. 

The data are tabulated in counts and are presented in the format shown in table 4. 

The first two segments, marked a and g at the bottom of table 5, are the x- and 
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y-coordinate readings for the pipper position (PIPD); the next two segments, marked 
c and d, are the x- and y-coordinate readings for the target position (TARGD); these , 
four segments are tabulated every other frame. The segments marked e and f in ta- 
ble 5 are the x- and y-coordinate readings for the left wingtip (TL.WD), while seg- 
ments g and h are the respective readings for the right wingtip (TRWD); these last 
four segments are tabulated every twenty-fourth frame. (For this tracking run, the 
wingtip coordinates were not recorded for the first frame.) The segment marked i in 
table 5 indicates that the film light event was off during this portion of the scored 
data. Segment j denotes that the aircraft type was the F-14, segment k denotes that 
the flight number was 564, segment 1 denotes that the film magazine number was 1 , 
segment m denotes that the tracking run number was 4, and segment n denotes that 
the frame sequence number for this portion of the scored data ranged from 0 to 120. 

Computation of Tracking parameters 

The first step in computing the tracking parameters is to determine the reticle 
calibration and the pipper reference point. The x and y coordinates at points A, B, 
C, and D on the reticle where the data are read from the three frames following the 
C-marked frame on the 2-sec film segment are tabulated in figure 10. The data for 
each frame are recorded on a single data card and only the x and y coordinates per- 
tinent to the reticle calibration are tabulated. The three sets of readings are 
averaged with respect to the errors in the reading and the recording of the data. 

The readings are then summed and averaged to determine the reticle calibration. 

The reticle calibration is obtained by determining the ratio of the calibrated 
reticle diameter RDM to the uncalibrated reticle diameter RD. The RDM is known from 
the reticle specifications; the RD is calculated using the three radius segments AB, 
BC, and DB (fig. 10) to average the reticle distortion caused by projection. Thus, 

0.5RD = ( ( AXav - BXav) + (BXav - CXav) + (DYav - BYav) ) / 3.0 (1) 

given that RDM = 120 mils, RD = 2167.8 counts, and the calibration is 
0.0553 mils/count. 

The pipper reference point is defined by the average of x and y coordinates at 
point B. Thus, the pipper azimuth reference (on the x coordinate) is 

PREFX = BXav = 2754.3 counts, 

and the pipper elevation reference (on the y coordinate) is 

PREFY = BYav = 2237.7 counts. 

Figure 11 shows a frame containing the target errors relative to the pipper 
( MDX, MDY, MDR) , the pipper position relative to the pipper reference point (PMDX, 
PMDY, PMDR) , the reticle diameter (RD), the apparent target aircraft wingspan (SP), 
and pipper reference point (PRP) . The pipper position on figure 11 is shown for an 
arbitrary instant during the tracking run within the 100-sec film segment. Although 
the pipper is constrained to move parallel to the y axis, the lateral offset of the 
pipper in relation to the pipper reference point PMDX = 0 is shown offset for illus- 
tration purposes only. For purposes of clarity, the reticle and scale are enlarged 
in figure 1 1 . 
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The input parameters used for computing the tracking parameters are given in the 
first two lines of table 5. The initial start time 1ST is the first time point in 
which the camera is turned on; the 1ST is determined from the time correlation be- 
tween film data and PCM data. The reticle calibration parameters RD, RDM, PREFX, 
and PREFY are obtained from the reticle calibration. The difference between what 
the pilot observes and what the camera photographs are the target bias parameters. 
Both the target azimuth (x coordinate) bias TBX and the target elevation (y coor- 
dinate) bias TBY are zero because the optical sight and the camera are mounted on 
a single mount with an exact alignment. 

The tracking errors at a given instant during the run are computed by calculat- 
ing the relative x and y distances between the target and the pipper, correcting 
for target bias, and converting the raw units in counts to engineering units in 
mils. Equations (2) to (4) are used to compute the target tracking error in azimuth 
(x coordinate) MDX, the target tracking error in elevation (y coordinate) MDY, and 
for target radial tracking error MDR: 


MDX = ((KTX - KPX) 

- TBX ) * 

(RDM / 

RD) 

(2) 

MDY = ((KPY - KTY) 

- TBY) * 

(RDM / 

RD) 

(3) 

MDR = SQRT (MDX * 

MDX + MDY 

* MDY) 


(4) 


where 

KTX is x coordinate of target image position ( TARGD ) , 

KPX x coordinate of the pipper image position (PIPD), 

TBX x coordinate of the target azimuth bias, 

KPY y coordinate of the pipper image position (PIPD), 

KTY y coordinate of the target image position (TARGD), 

TBY y coordinate of the target elevation bias, and 

SQRT square root. 

The pipper position relative to the pipper reference point at the same instant 
during the run is computed by calculating the x- and y-coordinate distances between 
the pipper and pipper reference point and converting the raw units to engineering 
units. Equations (5) to (7) are used to compute the pipper position in azimuth 
PMDX, elevation PMDY, and radial PMDR, respectively: 


PMDX = (KPX 

- PREFX) 

* (RDM / RD) 

(5) 

PMDY = (KPY 

- PREFY) 

* (RDM / RD) 

(6) 

PMDR = SQRT 

(PMDX * 

PMDX + PMDY * PMDY) 

(7) 
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An approximate value of the target aircraft range at a rate of 1 sample/sec is 
computed as follows.- 

RANGE = 0.5 * WGSPN / ((SP/ RD) TAN (0.0005 * RDM)) (8) 

where WGSPN is the actual wingspan of the target aircraft. The parameter SP is the 
apparent wingspan of the target aircraft, obtained by calculating the apparent x- 
and y-coordinate lengths between the target aircraft left and right wingtips and 
computing the resultant of these lengths. Equations (9) to (11) are used to per- 
form this operation: 

SPX = KTRWX - K.TLWX (9) 

SPY = KTLWY - KTRWY (10) 

SP = SQRT (SPX * SPX + SPY * SPY) (11) 

where KTLWX and KTLWY are the field names for the x and y coordinates, respec- 
tively, of the target left wingtip image position. Similarly, KTRWX and KTRWY 
are the respective field names for the right wingtip image position. 

Table 6 shows a partial tabulation of the computed tracking parameters for a 
typical tracking run. The data are presented at 1 2 samples/sec for the tracking 
errors (MDX, MDY, and MDR) , SP/RD ratio, target range and pipper positions (PMDX, 
PMDY, and PMDR) . The target range is zero for the first second because no target 
wingtip data were recorded for the first frame. Also, PMDX is very close to zero, 
and the pipper motion is primarily along the y axis as desired. 


RESULTS AND DISCUSSION 


Figure 12 presents a flight time history of a 3-g tracking maneuver and is an 
example of the data used for analysis. In figure 12(a), the pipper motion (PMDY) 
provided by the movement of the combining glass was nearly identical to the input 
signal from the optical sight drive motor (OSDM) . As indicated in figure 12(b), 
although the pipper was driven along the y axis in pitch, the pilot used longitu- 
dinal and lateral stick and occasionally rudder pedals to track the target. This 
resulted in tracking errors in both azimuth (MDX) and elevation (MDY) in fig- 
ure 12(a). The pilot was given the option of using the rudder pedals to aid in 
tracking. Specific pilot comments on the use of rudder pedals when tracking in 
flight are summarized in appendix B. 

For this particular run, 42 sec after the maneuver began, the pilot allowed the 
target range to slowly increase to approximately 3300 ft. Also, the computed target 
range during the first second of the maneuver was set to zero because no data was 
obtained to compute this parameter for this period of the run. 

Although quantitative and complete results of this experiment are not within the 
scope of this report, the flight data are available for follow-on analysis by calcu- 
lating the mean and standard deviations for selected aircraft and tracking parame- 
ters. The quantitative results can be used for comparison with results from the 
pilot-vehicle system model predictions for the F-14 airplane. 
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Each pilot was given a questionnaire to complete after flying the Langley DMS 
and VMS simulators and the F-14 aircraft at Ames-Dryden. A summary of pilot com- 
ments is given in appendix B. 

In general, lateral-directional sluggishness was not a problem when the pilots 
flew the F-14 aircraft, although one pilot commented that the aircraft was slightly 
sluggish in the lateral axis. The pilots commented that the major difficulty in 
tracking was due to turbulence caused by flying into the target aircraft wake. 
Specifically, they commented that difficulties in tracking with wings level were 
caused by dutch-roll dynamics and wake encounters especially behind the F-104 target 
aircraft. One pilot commented that the target wake was encountered only once during 
the tracking at 3 g. The pilots generally favored the use of rudder pedal inputs 
while tracking. Specific comments indicated that some pilots used no rudder pedal 
inputs, some used rudder pedals only while tracking with wings level, and some used 
rudder pedals only to make small corrections while tracking. Some pilots believed 
that the best tracking occurred when rudder pedal input was used only for lateral- 
directional error corrections; other pilots indicated that tracking would be 
impossible without rudder pedal input. 


CONCLUDING REMARKS 


The Dryden Flight Research Facility of NASA Ames Research Center (Ames-Dryden) 
and NASA Langley Research Center conducted a joint experiment to investigate simu- 
lation validation cue fidelity. The primary contribution of Ames-Dryden was the 
development of a method for analyzing flight data for closed pilot-in-the-loop anal- 
ysis. This method consists of scoring and digitizing raw data from film obtained 
with an optical gunsight and a high-quality 16 mm movie camera. These raw data were 
used to compute basic tracking parameters and were merged with pulse-code modulation 
data to create a single data file — the first time a combined data file was created 
in this manner at Ames-Dryden. Although the data will be used primarily for com- 
parison with results from the pilot-vehicle model predictions, the flight data 
constitute. a valuable data base for the determination of pilot models in a real 
flight environment. 
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APPENDIX A — PILOT QUESTIONNAIRE FOR F-14 FLIGHT 


1. Mark the level of difficulty you had in keeping the pipper on the target during 
the task. 


Very Very 

easy difficult 


0123456789 10 

2. In which axis did the difficulties occur? Check the appropriate reasons: 

Longitudinal axis 

Longitudinal-control 
Short-period dynamics 

Time-delay response (for example, sluggishness) 

Lateral axis 

Lateral-directional control 
Dutch-roll dynamics 
Adverse yaw 
Wing rock 

Time-delay response 

3. Did you use the rudder pedals to help you perform the tracking task? 

4. Rank the following parameters in terms of their importance in providing infor- 
mation for the tracking task. 

Normal acceleration 

Lateral acceleration 

Aircraft roll, pitch, and yaw rates 

Rotational accelerations in roll, pitch, and yaw 

Longitudinal acceleration 

5. Were there any other factors that you believe were important in performing the 
tasks but that were not covered in this questionnaire? 
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APPENDIX B — SUMMARY OF PILOT COMMENTS 


Difficulties in keeping the pipper on the target during the tracking task were 
caused by dutch-roll dynamics and encounters with the target aircraft wake. The 
F-14 aircraft is usually somewhat sluggish in the lateral axis. 

Wake turbulence from the F-104 target aircraft was encountered while tracking 
with the wings level at 200 knots indicated airspeed; this made tracking difficult. 

Lateral-directional control and adverse yaw caused difficulties in keeping the 
pipper on the target while tracking. 

There seemed to be a problem with the pipper driving to the top of the head-up 
display and limiting on the upper stop. 

Because of jet wake, especially behind the F-104 target aircraft, the tracking 
task with wings level was difficult to perform. Jet wake was encountered only once 
when tracking at 3 g. 

No inherent aircraft factors were noted which effected the performance of the 
F-14 aircraft. However, the data obtained when tracking with wings level at 200 
KIAS was heavily contaminated by wake encounters from the F-104 target aircraft. 

Used rudder pedals during tracking. 

Used rudder pedals for small corrections. 

Did not use rudder pedals during tracking. 

Used rudder pedals for fine tracking. 

Did not use rudder pedals while tracking. 
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TABLE 1. - FREQUENCY AND AMPLITUDE 
OF GENERATED SINE WAVES 


Frequency, 

rad 

Harmonics 

Relative 

amplitude 

0. 147 

2 

i 

1,17000 

0.442 

6 

0.72600 

0.957 

13 

0.29300 

1.399 

19 

0.12700 

1.988 

27 

0.07530 

2.798 

38 

0.04570 

3.976 

54 

0.02760 

5.670 

77 

•0.01630 

7.952 

108 

0.00955 

10.971 

149 

0.00592 

15.978 

217 

0.00359 


TABLE 2. — SIMULATOR PILOT RATING SCALE 



Category: 

Satisfactory representation of actual vehicle 

Rating 

Adjective 

Description 

1 

Excellent 

Virtually no discrepancies: simulator reproduces actual 

vehicle characteristics to the best of my memory. Simu- 
lator results directly applicable to actual vehicle with 
high degree of confidence. 

2 

Good 

Very minor discrepancies. The simulator comes close to 
duplicating actual vehicle characteristics. Simulator 
results in most areas would be applicable to actual 
vehicle with confidence. 

3 

Fair 

Simulator is representative of actual vehicle. Some minor 
discrepanices are noticeable but not distracting enough 
to mask primary characteristics. Simulator trends could 
be applied to actual vehicle. 


Category: Unsatisfactory representation of actual vehicle 


4 


5 


6 


Poor 


Bad 


Very bad 


Simulator needs work. It has many minor discrepancies which 
are annoying. Simulator would need some improvement 
before applying results directly to actual vehicle but is 
useful for general handling-qualities investigations for 
this class of aircraft. 

Simulator not representative. Discrepancies exist which pre 
vent actual vehicle characteristics from being recognized. 
Results obtained here should be considered as unreliable. 

Possible simulator malfunction. Wrong sign, inoperative 
controls, other gross discrepancies prevent comparison 
from even being attempted. No data. 


Table 2 taken from Szalai (1981). 
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TABLE 3. — SIMULATOR CONFIGURATION TEST MATRIX 


DMS, six-degree-of-f reedom nonlinear model, narrow field of view 
No cues 
G-seat only 
Helmet loader only 
G-seat and helmet loader 

DMS, six-degree-of-f reedom nonlinear model, wide field of view 
No cues 
G-seat only 
Helmet loader only 
G-seat and helmet loader 

DMS, three-degree-of -freedom linear model, narrow field of view 
No cues 

DMS, three-degree-of-f reedom linear model, wide field of view 
No cues 

VMS, six-degree-of-f reedom nonlinear model, narrow field of view 
No cues 
G-seat only 
Platform motion only 
G-seat and platform motion 


'TABLE 4. — FORMAT FOR RECORDING 
SCORED FILM DATA 


Field 

name 

Card 

column 

FORTRAN data 
format 

KPX 

11-15 

F5.0 

KPY 

16-20 

F5.0 

KTX 

21-25 

F5.0 

KTY 

26-30 

F5.0 

KTLWX 

31-35 

F5.0 

KTLWY 

36-40 

F5.0 

KTRWX 

41-45 

F5.0 

KTRWY 

46-50 

F5.0 

ICODE 

56-57 

12 

= 01 

reticle calibration 

= 02 

frame cannot be scored 

= 03 

film light 

event-off 

= 04 

film light 

event-on 

ACRFT 

59-61 

13 

FLTN 

62-64 

A3 

FILMN 

65-66 

12 

RUNUM 

67-68 

12 

FRAMEN 

69-73 
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TABLE 5. -TABULATION OF SCORED FILM DATA 


$NAM1 I ST- 13, 40. 12. 457, RD-2167.8, RDM-120.00, TBX-00.00, TBY-00.00, 
PREFX-2754 . 3 , PREFY-2237 .7 , GCFR-12.0, WGSPN-25.2 SEND 


2750 

2614 

2628 

2592 

0000 

0000 

0000 

0000 

3 

014564010400000 

2757 

2612 

2647 

2601 

0000 

0000 

0000 

0000 

3 

014564010400002 

2768 

2621 

2673 

2617 

0000 

0000 

0000 

0000 

3 

014564010400004 

2756 

2620 

2671 

2610 

0000 

0000 

0000 

0000 

3 

014564010400006 

2749 

2639 

2684 

2627 

0000 

0000 

0000 

0000 

3 

014564010400008 

2758 

2653 

2699 

2633 

0000 

0000 

0000 

0000 

3 

014564010400010 

2745 

2674 

2687 

2640 

0000 

0000 

0000 

0000 

3 

014564010400012 

2748 

2689 

2685 

2651 

0000 

0000 

0000 

0000 

3 

014564010400014 

2761 

2707 

2691 

2649 

0000 

0000 

0000 

0000 

3 

014564010400016 

2744 

2719 

2674 

2636 

0000 

0000 

0000 

0000 

3 

014564010400018 

2755 

2753 

2670 

2673 

0000 

0000 

0000 

0000 

3 

014564010400020 

2760 

2773 

2659 

2683 

0000 

0000 

0000 

0000 

3 

014564010400022 

2764 

2792 

2653 

2706 

2524 

2707 

2801 

2707 

3 

014564010400024 

2755 

2814 

2634 

2736 

0000 

0000 

0000 

0000 

3 

014564010400026 

2764 

2836 

2639 

2758 

0000 

0000 

0000 

0000 

3 

014564010400028 

2764 

2849 

2636 

2768 

0000 

0000 

0000 

0000 

3 

014564010400030 

2750 

2868 

2615 

2791 

0000 

0000 

0000 

0000 

3 

014564010400032 

2757 

2907 

2627 

2818 

0000 

0000 

0000 

0000 

3 

014564010400034 

2756 

2893 

2631 

2816 

0000 

0000 

0000 

0000 

3 

014564010400036 

2750 

2930 

2626 

2826 

0000 

0000 

0000 

0000 

3 

014564010400038 

2764 

2954 

2650 

2854 

0000 

0000 

0000 

0000 

3 

014564010400040 

2747 

2963 

2643 

2848 

0000 

0000 

0000 

0000 

3 

014564010400042 

2746 

2987 

2648 

2869 

0000 

0000 

0000 

0000 

3 

014564010400044 

2763 

2971 

2669 

2848 

0000 

0000 

0000 

0000 

3 

014564010400046 

2749 

3075 

2664 

2876 

2542 

2862 

2796 

2869 

3 

014564010400048 

2751 

3043 

2657 

2890 

0000 

0000 

0000 

0000 

3 

014564010400050 

2748 

3046 

2655 

2893 

0000 

0000 

0000 

0000 

3 

014564010400052 

2749 

3054 

2644 

2931 

0000 

0000 

0000 

0000 

3 

014564010400054 

2750 

3043 

2649 

2947 

0000 

0000 

0000 

0000 

3 

014564010400056 

2750 

3037 

2629 

2980 

0000 

0000 

0000 

0000 

3 

014564010400058 

2752 

3026 

2622 

3004 

0000 

0000 

0000 

0000 

3 

014564010400060 

2752 

3040 

2610 

3032 

0000 

0000 

0000 

0000 

3 

014564010400062 

2744 

3048 

2595 

3058 

0000 

0000 

0000 

0000 

3 

014564010400064 

2764 

3052 

261 1 

3087 

0000 

0000 

0000 

0000 

3 

014564010400066 

2766 

3047 

2608 

3080 

0000 

0000 

0000 

0000 

3 

014564010400068 

2760 

3046 

2606 

3079 

0000 

0000 

0000 

0000 

3 

014564010400070 

2758 

3048 

2613 

3073 

2496 

3059 

2736 

3070 

3 

014564010400072 

2757 

3037 

2612 

3066 

0000 

0000 

0000 

0000 

3 

014564010400074 

2745 

3044 

2622 

3062 

0000 

0000 

0000 

0000 

3 

014564010400076 

2752 

3014 

2626 

3027 

0000 

0000 

0000 

0000 

3 

014564010400078 

2757 

3036 

2656 

3051 

0000 

0000 

0000 

0000 

3 

014564010400080 

2756 

3005 

2653 

3028 

0000 

0000 

0000 

0000 

3 

014564010400082 

2747 

2991 

2670 

3011 

0000 

0000 

0000 

0000 

3 

014564010400084 

2751 

2985 

2681 

3019 

0000 

0000 

0000 

0000 

3 

014564010400086 

2752 

2978 

2692 

3016 

0000 

0000 

0000 

0000 

3 

014564010400088 

2754 

2982 

2703 

3018 

0000 

0000 

0000 

0000 

3 

014564010400090 

2754 

2972 

2710 

3007 

0000 

0000 

0000 

0000 

3 

014564010400092 

2747 

2963 

2706 

2997 

0000 

0000 

0000 

0000 

3 

014564010400094 

2761 

2959 

2719 

2993 

2610 

2984 

2838 

2989 

3 

014564010400096 

2758 

2954 

2709 

2988 

0000 

0000 

0000 

0000 

3 

014564010400098 

2758 

2945 

2704 

2976 

0000 

0000 

0000 

0000 

3 

014564010400100 

2745 

2908 

2697 

2953 

0000 

0000 

0000 

0000 

3 

014564010400102 

2747 

2929 

2690 

2968 

0000 

0000 

0000 

0000 

3 

014564010400104 

2762 

2907 

2684 

2954 

0000 

0000 

0000 

0000 

3 

014564010400106 

2758 

2899 

2665 

2967 

0000 

0000 

0000 

0000 

3 

014564010400108 

2759 

2892 

2655 

2962 

0000 

0000 

0000 

0000 

3 

014564010400110 

2756 

2885 

2642 

2967 

0000 

0000 

0000 

0000 

3 

014564010400112 

2760 

2891 

2636 

2972 

0000 

0000 

0000 

0000 

3 

014564010400114 

2750 

2885 

2614 

2966 

0000 

0000 

0000 

0000 

3 

014564010400116 

2761 

2861 

2620 

2955 

0000 

0000 

0000 

0000 

3 

014564010400118 

2765 

2855 

2620 

2954 

2506 

2938 

2741 

2944 

3 

014564010400120 

1 a 

1 b 

1 c 

1 d 

1 e 

1 f 1 

g 1 

h 1 

| i | j | k I 1 |m| n I 



TABLE 6. — TABULATION OF COMPUTED TRACKING PARAMETERS 



TIME 


MDX 

MDY 

MDR 

SP/RD 

RANGE 

PMDX 

PMDY 

PMDR 

13 

40 

12 

457 

- 6.75 

-1 .22 

6.86 

.000 

0 . 

-.24 

20.83 

20.83 

13 

40 

12 

540 

- 6.09 

-.61 

6.12 

.000 

0 . 

.15 

20.72 

20.72 

13 

40 

12 

623 

- 5.26 

-.22 

5.26 

.000 

0 . 

.76 

21 .22 

21 .23 

13 

40 

12 

706 

- 4.71 

-.55 

4.74 

.000 

0 . 

.09 

21.16 

21.16 

13 

40 

12 

790 

- 3.60 

-.66 

3.66 

.000 

0 . 

-.29 

22.21 

22.22 

13 

40 

12 

873 

- 3.27 

- 1.11 

3.45 

.000 

0 . 

.20 

22.99 

22.99 

13 

40 

12 

956 

- 3.21 

-1 .88 

3.72 

.000 

0 . 

-.51 

24.15 

24.16 

13 

40 

13 

40 

- 3.49 

- 2.10 

4.07 

.000 

0 . 

-.35 

24.98 

24.98 

13 

40 

13 

123 

- 3.87 

- 3.21 

5.03 

.000 

0 . 

.37 

25.98 

25.98 

13 

40 

13 

206 

- 3.87 

- 4.59 

6.01 

.000 

0 . 

-.57 

26.64 

26.65 

13 

40 

13 

290 

- 4.71 

- 4.43 

6.46 

.000 

0 . 

.04 

28.52 

28.52 

13 

40 

13 

373 

- 5.59 

- 4.98 

7.49 

.000 

0 . 

.32 

29.63 

29.63 

13 

40 

13 

456 

- 6.14 

- 4.76 

7.77 

.128 

1641 . 

.54 

30.68 

30.69 

13 

40 

13 

540 

- 6.70 

- 4.32 

7.97 

.128 

1641 . 

.04 

31 .90 

31 .90 

13 

40 

13 

623 

- 6.92 

- 4.32 

8.16 

.128 

1641 . 

.54 

33.12 

33.12 

13 

40 

13 

706 

- 7.09 

- 4.48 

8.39 

.128 

1641 . 

.54 

33.84 

33.84 

13 

40 

13 

790 

- 7.47 

- 4.26 

8.60 

.128 

1641 . 

-.24 

34.89 

34.89 

13 

40 

13 

873 

- 7.20 

- 4.93 

8.72 

.128 

1641 . 

.15 

37.05 

37.05 

13 

40 

13 

956 

- 6.92 

- 4.26 

8.13 

.128 

1641 . 

.09 

36.27 

36.27 

13 

40 

14 

40 

- 6.86 

- 5.76 

8.96 

.128 

1641 . 

-.24 

38.32 

38.32 

13 

40 

14 

123 

- 6.31 

- 5.54 

8.39 

.128 

1641 . 

.54 

39.65 

39.65 

13 

40 

14 

206 

- 5.76 

- 6.37 

8.58 

.128 

1641 . 

-.40 

40.15 

40.15 

13 

40 

14 

290 

- 5.42 

- 6.53 

8.49 

.128 

1641 . 

-.46 

41.48 

41 .48 

13 

40 

14 

373 

- 5.20 

- 6.81 

8.57 

.128 

1641 . 

.48 

40.59 

40.59 

13 

40 

14 

456 

- 4.71 

- 11 .02 

11 .98 

. 117 

1789 . 

-.29 

46.35 

46.35 

13 

40 

14 

540 

- 5.20 

- 8.47 

9.94 

.117 

1789 . 

-.18 

44.58 

44.58 

13 

40 

14 

623 

- 5.15 

- 8.47 

9.91 

. 117 

1789 . 

-.35 

44.74 

44.75 

13 

40 

14 

706 

- 5.81 

- 6.81 

8.95 

.117 

1789 . 

-.29 

45.19 

45.19 

13 

40 

14 

790 

- 5.59 

- 5.31 

7.71 

.117 

1789 . 

-.24 

44.58 

44.58 

13 

40 

14 

873 

- 6.70 

- 3.16 

7.40 

.117 

1789 . 

-.24 

44.25 

44.25 

13 

40 

14 

956 

- 7.20 

-1 .22 

7.30 

.117 

1789 . 

-.13 

43.64 

43.64 

13 

40 

15 

40 

- 7.86 

-.44 

7.87 

.117 

1789 . 

-.13 

44.41 

44.41 

13 

40 

15 

123 

- 8.25 

.55 

8.27 

.117 

1789 . 

-.57 

44.85 

44.86 

13 

40 

15 

206 

- 8.47 

1 .94 

8.69 

.117 

1789 . 

.54 

45.08 

45.08 

13 

40 

15 

290 

- 8.75 

1 .83 

8.93 

. 1 17 

1789 . 

. 65 ' 

44.80 

44.80 

13 

40 

15 

373 

- 8.52 

1 .83 

8.72 

.117 

1789 . 

.32 

44.74 

44.74 

13 

40 

15 

456 

- 8.03 

1 .38 

8.14 

.111 

1893 . 

.20 

44.85 

44.85 

13 

40 

15 

540 

- 8.03 

1 .61 

8.19 

.111 

1893 . 

.15 

44.25 

44.25 

13 

40 

15 

623 

- 6.81 

1 .00 

6.88 

.111 

1893 . 

-.51 

44.63 

44.64 

13 

40 

15 

706 
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Figure 1(a). Optical gunsight and movie camera mounted in F-14 
test aircraft . 
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Figure 1(b). Typical frame of 
movie film showing the reticle 
as displayed to the pilot. 
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Figure 2. Langley differential maneu- 
vering simulator. 


Figure 3. Langley visual motion 
simulator. 
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Figure 4(b). Operation of the g seat. 


Figure 4(a). A g-seat system. 
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Figure 5. Helmet loader installed in 
differential maneuvering simulator . 
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Ames-Dryden F-14 aircraft production model IX. 
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Figure 7(a). Optical gunsight power, 
trim, and reset controls mounted in 
pilot's right-hand console above the 
master generator panel . 


Figure 7(b). Optical gunsight pipper 
depression angle. 
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Figure 8. Preparation of flight data 
for analysis . 


Figure 9. Typical frame of film 
as seen on screen showing basic 
scoring parameters . 
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Ax, 

counts 
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counts 
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Cx, 

counts 

Dy. 

counts 

1 

3832.0 

2744.0 

2227.0 

1659.0 

3299.0 

2 

3851.0 

2757.0 

2245.0 

1664.0 

3312.0 

3 

3852.0 

2762.0 

2241.0 

1671.0 

3316.0 

Total 

11535.0 

8263.0 

6713.0 

4994.0 

9927.0 

Average 

3845.0 

2754.3 

2237.7 

1664.7 

3309.0 
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Figure 10. Data for determining pipper 
calibration and pipper reference point. 
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